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Abstract 


This  proof-of-concept  case  study  analyzes  the  potential  benefits  of  open 
architecture  (OA)  in  the  AEGIS  software  maintenance  and  upgrade  process.  In  a 
multi-phased  approach,  the  Knowledge  value  Added/Real-Options  (KVA+RO) 
framework  was  applied  to  sustaining  engineering  on  specific  AEGIS  software 
processes. 

The  KVA+RO  framework  provides  decision-makers  with  a  systematic 
approach  for  analyzing  benefits  and  assessing  risks  of  potential  technological 
acquisitions.  Results  from  our  research  indicate  that  implementing  OA  could  result 
in  substantial  cost  savings,  optimal  return  on  investment  and  increased  benefits. 
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1.0 


Introduction 


The  US  Navy  (Navy)  must  meet  evolving  national  security  requirements  and 
build  a  fleet  for  the  future  in  a  time  of  shrinking  budgets  and  aging  platforms. 
Through  open  architecture  (OA)  principles  and  solutions,  Naval  systems  could 
become  more  agile,  modular,  flexible  and  affordable.  Naval  Open  Architecture,  an 
enterprise-wide,  multi-faceted  business  model  and  product-line  strategy,  is  designed 
to  exploit  open  system  design  principles  and  architectures. 

This  proof-of-concept  case  study  quantifies  the  potential  benefits  of  open 
architecture  in  the  AEGIS  software  maintenance  and  upgrade  process.  In  a  multi- 
phased  approach,  the  Knowledge-value  Added/Real-options  (KVA+RO)  framework 
was  applied  to  sustaining  engineering  in  the  AEGIS  software  maintenance  and 
upgrade  process.  The  KVA+RO  framework  provides  decision-makers  with  a 
systematic  approach  for  analyzing  benefits  and  assessing  risks  of  potential 
technological  acquisitions. 

This  case  study  augments  previous  research  by  the  Naval  Postgraduate 
School  (NPS)  and  analyzes  the  potential  impact  of  OA  from  the  warfighter 
perspective  by  extending  that  initial  research  into  the  development  and  acquisition 
processes  of  sustaining  engineering  (Uchytil,  2006).  By  extending  our  investigation 
into  the  acquirer  and  system-developer  perspective,  the  researchers  can  provide  a 
comprehensive  view  of  the  entire  system  development  lifecycle. 

In  the  first  phase,  KVA  methodology  was  first  applied  under  two  scenarios: 
As-is  and  To-be.  Results  from  our  KVA  analysis  show  that  implementing  OA  could 
result  in  substantial  cost  savings,  optimal  return  on  investment  and  increased 
benefits.  In  particular: 

Costs  for  one  ship  decrease  $365,105;  costs  for  all  ships  decrease  by 
$26,543,825  per  year. 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY  - 1  - 

NAVAL  POSTGRADUATE  SCHOOL 


Return  on  investment  for  one  ship  increases  from  69%  to  789%;  ROI 
for  all  ships  increases  from  320%  to  72,287%. 


■  Revenues  (benefits)  for  one  ship  increase  $2,488,179  to  $3,837,931; 
revenues  for  all  ships  increase  $209,007,032  to  $322,386,181 . 

During  Phase  two,  a  real-options  analysis  was  conducted  on  several  strategic 
scenarios  to  assess  risks  associated  with  potential  strategies  for  the  AEGIS  software 
maintenance  and  upgrade  process.  This  paper  presents  the  research  in  greater 
detail. 
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2.0 


Naval  Open  Architecture 


Naval  Open  Architecture  (OA)  is  a  multi-faceted  business  and  technical 
strategy  for  acquiring  and  maintaining  National  Security  Systems  as  interoperable 
systems  that  adopt  and  exploit  open  system  design  principles  and  architectures 
(Naval  OA  Strategy,  2007).  It  is  a  departure  from  the  old  acquisition  model  of 
purchasing  stove-piped  systems  built  for  single  uses,  not  designed  to  work  in  a 
networked  environment. 


Figure  1.  Past  and  Present  Navy  Enterprise  Acquisition  Models 

(Shannon,  2007,  May  9,  p  15) 


PAST- 


Business  Model  Attributes 

Platform  Focused 
Owner  controls  evolution 
Cost  emphasis 
Develop  software 
Make  custom  hardware 


System  Model  Attributes 

Requirements  driven 

Specification  focus 

Rigid  requirements 

Unique  /  monolithic  architectures 

Stable  design 

Ignore  evolution 

Obsolescence  Waterfall-style 

development 


PRESENT - 


Business  Model  Attributes:  System  Model  Attributes: 

Capability  /  Systems  Focused  Market  driven 
Market  controls  evolution  Business  plan  focus 

Total  Ownership  Cost  emphasis  Flexible  requirements 
License  or  Reuse  software  Modular  open  architectures 

Leverage  COTS  or  Reuse  Constant  changes 

Design  for  tech  refresh 
Early-managed  obsolescence 
Spiral  development 


OA  has  led  to  creation  of  interoperable  systems  delivering  improved 
capabilities  in  a  shorter  time  frame.  For  instance,  the  Naval  Air  Systems  Command 
Office,  responsible  for  the  E-2C  Hawkeye  command-and-control  aircraft,  faced  both 
delays  in  enhancing  capabilities  for  its  mission  computing  system  as  well  as 
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obsolescence  by  the  time  it  was  to  be  fielded.  By  adopting  OA  principles,  new 
capabilities  were  integrated  within  24  hours,  and  the  acquisition  cycle-time  was 
reduced  from  seven  years  to  two-and-a-half  (Shannon,  2007,  February).  The 
submarine  force  sonar  program  increased  performance  seven-fold,  cut  real 
processing  costs  60-fold  and  fielded  four  major  improvements  within  five  years 
(2007,  February). 

SOA  is  another  solution  the  Navy  enterprise  is  considering  in  building  future 
systems  beyond  OA.  A  new  shift  is  occurring  in  IT,  enabled  by  several  factors, 
including  processing  speed,  storage,  network  technology  and  new  business  models. 
In  the  past,  software  was  traditionally  developed  to  support  very  specific 
requirements  and  was  installed  at  very  specific  sites.  Software  has  shifted  to  a 
service-oriented  industry.  This  paradigm  change  will  have  a  major  impact  on  every 
organization. 

Software  is  an  increasingly  important  functionality  in  Naval  combat  systems. 
The  size  of  the  DDG  1000  combat  system,  for  example,  is  expected  to  increase  35% 
to  almost  1 .8  MSLOC — larger  than  the  AEGIS  Baseline  7.1  R  (Horvitz  E.,  Katz  D.J., 
Rumpf,  R.L,  Shrobe,  H.,  Smith,  T.B,  Webber,  G.E,  Williamson,  W.E.,  Winston,  P.H., 
Wolbarsht,  James  L.,  2006) 

Figure  2.  Size  of  Typical  Naval  Combat  System 

(Horvitz  ,  Katz,  Rumpf,  Shrobe,  Smith,  Webber,  Williamson,  Winston,  Wolbarsht, 

July  2006,  p  29) 


Aegis  B  L  7.1R  DDG  1000  F’A18  E'F  JSF 

Surface  Ship  Aircraft  Systems 

Combat  Systems 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


-4  - 


3.0 


Service-oriented  Architecture 


Service-oriented  architecture  is  a  business-driven  approach  providing 
increased  flexibility,  adaptability,  agility,  openness  and  cost-efficiency.  It  supports  an 
information  environment  built  upon  loosely  coupled,  reusable,  standards-based 
services.  SOA  promotes  data  interoperability  rather  than  application  interoperability, 
and  with  its  use,  capabilities  can  be  reused — not  recreated  every  time.  According  to 
Dr.  Margaret  Myers,  Principal  Director  for  the  DoD  Deputy  Chief  Information  Officer, 
SOA  ultimately  provides  the  ability  to  discover,  access  and  use  data  to  the  people 
that  need  it,  when  they  need  it. 


Figure  3.  Service-oriented  Software 

(Shannon,  2007,  May  9,  p  10) 


Enable  Governance 

Govern  services 
throughout  the  service 
lifecycle 


Encourage  Reuse 

Find  and  reuse  services 
for  building  blocks  for  new 
composite  applications. 


Enhance  Connectivity 

Enable  dynamic  and 
efficient  interactions 
between  services  at 
I  runtime 


Help  optimize 
service  performance 

Enable  enforcement  of 
policies.  Impact  analysis 


As  seen  in  Figure  4,  in  SOA,  business  applications  are  broken  down  into 
separate  functions  (services)  that  can  be  used  independently  of  applications  and 
computing  platforms.  Organizations  can  integrate  functions  or  create  new 
capabilities  as  “building  blocks”  when  individual  functions  within  applications  are 
available. 
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Figure  4.  SOA  Illustration 

(DiMare,  Jay  ,  2006,  p  4) 
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Source:  BM  Institute  lor  Business  Value. 


Research  firm  Gartner,  Inc.,  believes  SOA  will  be  used  in  more  than  50%of 
mission-critical  operational  applications  and  business  processes  designed  in  2007 
and  in  more  than  80  percent  by  2010  (cited  in  Govtech,  2007,  April  26).  In  addition, 
Research  2.0  predicts  that  SOA  will  become  status  quo  by  2015  (2007,  May  31). 
Today,  SOA  is  used  by  businesses  around  the  world  in  virtually  every  industry, 
including: 


10  of  the  world’s  largest  auto  manufacturers, 

8  of  the  world’s  1 0  largest  banks, 

9  of  the  world’s  10  largest  telecommunications  companies, 

8  of  the  world’s  10  largest  insurers,  and 

half  of  the  30  largest  electronic  companies.  (IBM,  2006,  November) 
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Companies  are  scrambling  to  gain  traction  in  the  rapidly  expanding  SOA 
space  with  industry  analysts  estimating  that  the  SOA  market  could  reach  as  much  as 
$160  billion.  The  SOA  engine  market  is  primed  to  grow  from  $1 .3  billion  in  2006  to 
$3.7  billion,  according  to  WinterGreen  Research  (WinterGreen,  2007,  May  16).  A 
SOA  engine  is  defined  by  WinterGreen  as  middleware  providing  a  repository  or 
process  implementation  for  reusable  code.  Engines  include  application  servers, 
repositories,  ESB,  XML  compression,  security  capability,  databases  and  mission- 
critical  messaging. 

IBM,  Hewlett-Packard,  Microsoft,  Salesforce.com  and  Oracle  all  offer  SOA- 
based  offerings;  IBM  dominates  the  SOA  infrastructure  landscape  with  53  percent  of 
the  market  and  implementations  for  4,500  customers  (Lawson,  2007,  May  23).  As 
seen  in  Figure  5,  Microsoft  is  second  with  8  percent;  SUN,  SAP  and  Oracle, 
webMethods  and  TIBCO  all  hold  3  percent  of  the  market,  while  Sybase  and  BEA 
Systems  each  hold  2  percent  (2007,  May  23).  IBM  is  the  de  facto  industry-standard 
market  leader  given  its  depth  of  SOA  offerings,  client  implementations,  SOA-related 
patents  and  market  share. 
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Figure  5.  Worldwide  Services-oriented  Architecture  (SOA)  Engine  and 
Collaboration  License,  Services  and  Maintenance  Market  Shares,  2006 

(Wintergreen,  2007,  p  4) 
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Saugatuck  Technology  believes  the  industry  will  evolve  in  three  waves  and  is 
in  the  middle  of  a  13-year  adoption  cycle.  This  projection  is  detailed  in  Table  1 . 
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Table  1.  Three  Waves  of  SOA 

(Taft,  2007,  May  28,  p.  11) 


SOA  Features 

Wave  1 

Wave  II 

Wave  III 

(1999-2005) 

(1999-2005) 

(1999-2005) 

Deployment  Focus 

Departmental 
initiatives;  project- 
based 

Cross-departmental; 

process-based 

Enterprise  wide; 

Program-based 

Vendor  Value 
Proposition 

Integration  via  SOA 
and  Web  services 

SOA  enables 
business  process 
flexibility 

SOA  as  a 
foundational  IT 

resource 

Representative 

Projects 

Portals;  application  and 
data  integration 

Sharing  of  services; 
cross-departmental 
process  workflows 

End-to-end  business 
processes;  composite 
applications 

Although  SOA  is  still  in  the  early  adopter  stage,  companies  such  as 
Wachovia,  Cardinal  Health  and  Royal  Caribbean  were  among  the  companies 
discussing  successful  SOA  implementations  at  a  May  2007  IBM  conference.  At 
Cardinal  Health,  an  $81  billion  global  provider  of  health  care  products  and  services, 
SOA  helped  achieve  a  forty-fold  boost  in  productivity.  Processes  that  typically  took 
the  company  1 ,200  hours  to  perform  required  only  30  hours  when  using  SOA 
technology  (Taft,  2007,  May  28,  p.  11).  For  the  most  part,  early  adopters  of  SOA  are 
service  providers  that  have  traditionally  leveraged  new  technologies  to  differentiate 
themselves  from  the  competition.  Banks,  insurers,  engineers,  telcos,  and  other 
businesses  have  automated  and  re-automated  entire  business  process  flows  again 
and  again  (Research  2.0,  2007,  May  31).  Figure  6  illustrates  how  an  IT  architecture 
changes  after  adopting  SOA. 
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Figure  6.  Before  and  After  SOA 
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(Source:  Enterprise  Solutions  Compentency  Center,  U.S.  Army  PEO  EIS  &  Software 
Engineering  Center-Belvoir) 
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4.0 


The  KVA+RO  Valuation  Framework 


KVA+RO  is  a  comprehensive  measurement  process  and  an  integrated  tool  set 
developed  by  NPS  to  measure  and  evaluate  the  total  value  of  Naval  acquisitions.  It 
captures  data  across  a  spectrum  of  organizations  to  compare  returns  on 
investments,  outputs,  processes,  capabilities,  risks,  strategic  alternatives,  costs,  and 
value  (i.e.,  comparable  revenue).  KVA+RO  analytically  quantifies  uncertainty  and 
risk  elements  inherent  in  predicting  the  future,  includes  ways  to  mitigate  these  risks 
through  strategic  options  with  analysis  of  alternatives,  and  analytically  develops  and 
allocates  budgets  to  optimize  project  portfolios. 
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Figure  7.  KVA+RO  Valuation  Framework 
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Risks  and  Projects 
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Data 
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Large,  complex  organizations  (ranging  from  publicly  traded  Fortune  500  firms  to 
public-sector  entities)  can  use  the  KVA+RO  framework.  Its  focus  on  core  processes, 
sub-processes,  and  outputs  provides  several  advantages: 

■  Quantifies  value  of  specific  processes,  functions,  departments, 
divisions,  or  organizations  in  common  units, 

■  Provides  historical  data  on  costs  and  revenues  of  specific  processes 
and  tasks  of  specific  programs  or  organizations, 

■  Facilitates  regulatory  compliance  in  public-sector  legislation — such  as 
the  Clinger-Cohen  Act  of  1996  mandating  portfolio  management  for  all 
federal  agencies, 
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■  Highlights  operational  efficiencies/inefficiencies,  and 

■  Leverages  current  and  potential  portfolio  investments  by  estimating 
potential  total  value  created. 

By  utilizing  the  KVA+RO  framework,  organizations  can  drill  down  to  understand 
specific  processes  involved  in  the  production  of  an  output,  the  cost  of  each  process 
and  its  contribution  to  the  bottom  line.  Government  entities  can  use  the  framework 
to  enhance  existing  performance  tools;  on  the  corporate  side,  industry  can  employ 
the  framework  to  value  specific  divisions  or  operating  units  to  determine  division 
profitability  or  shareholder  value. 

The  KVA+RO  framework  has  been  used  in  a  variety  of  NPS  analyses, 
including: 

1 .  Shipyard  Maintenance 

For  one  specific  area  of  shipyard  planning  for  maintenance  alterations, 
the  framework  was  projected  to  increase  cost  savings  to  exceed  $40 
million  per  year  and  to  drastically  reduce  manpower  requirements 
using  commercial-off-the-shelf,  three-dimensional 
scanning/visualization  technology  and  collaborative  PLM  technology. 
(Komoroski,  C.,  Housel,  T.,  Horn,  S.,  &  Mun,  J.,  2006,  October) 

2.  AEGIS  and  Ship  Self-defense  (SSDS)  Platforms  Track  Management 

For  certain  elements  of  track  management  processes,  upgrading 
existing  IWS  functionality  were  projected  to  have  ROIs  ranging  from 
212%  to  404%  for  the  AEGIS  platform.  For  the  SSDS  platform,  ROIs 
were  also  significant.  (Uchytil,  J.,  Housel,  T.,  Horn,  S.,  &  Mun,  J., 
Tarantino,  E.,  2006,  October) 

The  KVA+RO  framework  is  also  being  implemented  in  SPAWAR  and  in  the  Army 
Rapid  Equipping  Force  project.  It  is  being  used  in  both  projects  to  improve 
processes,  reduce  cycle-times  and  costs,  and  increase  value.  It  both  allows  Navy 
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executives  to  acquire  intelligence  systems  via  a  portfolio  approach  as  well  as 
distributies  capabilities  to  Army  troops  in  the  field  (i.e.,  Iraq  and  Afghanistan)  very 
quickly  through  new  rapid  acquisition  processes.  Key  framework  components  are 
discussed  further  below. 

4.1  KVA+RO  Framework:  Knowledge-value  Added 
Methodology 

KVA  measures  the  value  provided  by  human  capital  assets  and  IT  assets  by 
analyzing  an  organization,  process  or  function  at  the  process-level.  It  provides 
insights  into  each  dollar  of  IT  investment  by  monetizing  the  outputs  of  all  assets — 
including  intangible  knowledge  assets.  By  capturing  the  value  of  knowledge 
embedded  in  an  organization’s  core  processes,  employees  and  IT,  KVA  identifies 
the  actual  cost  and  revenue  of  a  product  or  service.  Because  KVA  identifies  every 
process  required  to  produce  an  output  and  the  historical  costs  of  those  processes, 
unit  costs  and  unit  prices  of  products  and  services  are  calculated.  An  output  is 
defined  as  the  end-result  of  an  organization’s  operations;  it  can  be  a  product  or 
service,  as  shown  in  Figure  8. 


Figure  8  .  Measuring  Output 


For  the  past  15  years,  KVA  has  been  applied  in  over  100  organizations  in  the 
public  and  private  sectors,  ranging  in  size  from  under  20  employees  to  thousands. 
The  methodology  has  been  applied  in  35  areas  within  the  DoD,  from  flight  simulation 
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applications  to  maintenance  and  modernization  processes.  As  a  performance  tool, 
the  methodology: 


Compares  all  processes  in  terms  of  relative  productivity, 

Allocates  revenues  to  common  units  of  output, 

Measures  value  added  by  IT  by  the  outputs  it  produces, 

Relates  outputs  to  cost  of  producing  those  outputs  in  common  units, 
and 

Provides  common  units  of  measure  for  organizational  productivity. 


By  describing  processes  in  common  units,  the  methodology  also  permits 
market  comparable  data  to  be  generated;  this  ability  is  particularly  important  for  non¬ 
profits  like  the  Navy.  Using  a  Market  Comparable  approach,  data  from  the 
commercial  sector  can  be  used  to  estimate  price  per  common  unit,  allowing  for 
revenue  estimates  of  process  outputs  for  non-profits.  The  Market  Comparable 
approach  also  provides  a  common-units  basis  with  which  organization  can  define 
benefit  streams,  regardless  of  the  process  analyzed. 

KVA  differs  from  other  nonprofit  ROI  models  because  it  allows  for  revenue 
estimates,  enabling  the  use  of  traditional  accounting,  financial  performance  and 
profitability  measures  at  the  sub-organizational  level.  Figure  9  shows  differences 
between  traditional  accounting  and  process-based  accounting  methods. 
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Figure  9  .  Comparison  of  Traditional  Accounting  versus  Process-based 
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KVA  also  ranks  processes  by  the  degree  to  which  they  add  value  to  the 
organization  or  its  outputs.  This  assists  decision-makers  in  identifying  what 
processes  really  add  value — those  that  will  best  accomplish  a  mission,  deliver  a 
service,  or  meet  customer  demand.  Value  is  quantified  in  two  key  metrics:  Return 
on  Knowledge  (ROK)  and  Return  on  Knowledge  Investment  (ROI).  Calculations  of 
these  key  metrics  are  shown  in  Table  2. 

Table  2.  KVA  Metrics 


Metric 

Description 

Type 

Calculation 

Return  on  Knowledge  (ROK) 

Basic  productivity,  cash¬ 
flow  ratio 

Sub-corporate, 
process  level 
performance 
ratio 

Outputs-benefits  in 

common  units/cost  to 
produce  the  output 

Return  on  Investment  (ROI) 

Same  as  ROI  at  the 
sub-corporate,  process 
level 

Traditional 
investment 
finance  ratio 

(Revenue-investment 
cost)/investment  cost 

4.2  KVA+RO  Framework:  Real-options  Analysis 

Potential  strategic  investments  can  then  be  evaluated  with  real-options 
analysis  based  on  KVA  data.  The  analysis  applied  is  a  robust  and  analytical  process 
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incorporating  risk  identification  (applying  various  sensitivity  techniques),  risk 
quantification  (applying  Monte  Carlo  simulation),  risk  valuation  (applying  real-options 
analysis),  risk  mitigation  (utilizing  real-options  framing),  and  risk  diversification 
(employing  analytical  portfolio  optimization).  Figure  10  reflects  the  complex 
calculations  required  for  integrated  risk  analysis  in  the  KVA+RO  framework 
developed  by  Dr.  Jonathan  Mun. 

Figure  10.  Integrated  Risk  Analysis  Approach 
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project 
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. .  .the  user  generates  a  traditional 
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. .  .sensitivity  and  scenario  analysis 
coupled  with  Monte  Carlo  simulation  is 
added  to  the  analysis  and  the  financial 
model  outputs  become  inputs  into  the  real 
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. .  .stochastic  optimization  is  the  next 
optional  step  if  multiple  projects  exist  that 
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strategic  portfolio  management... 
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Real-options  analysis  incorporates  strategic  planning  and  analysis,  risk 
assessment  and  management,  and  investment  analysis.  As  a  financial  valuation 
tool,  Real-options  allows  organizations  to  adapt  decisions  to  respond  to  unexpected 
environmental  or  market  developments.  As  a  strategic  management  tool,  real- 
options  is  a  strategic  investment  valuation  tool  affording  decision-makers  the  ability 
to  leverage  uncertainty  and  limit  risk.  Real-options  can  be  used  to: 

■  Identify  different  corporate  investment  decision  pathways  or  projects 
that  management  can  consider  in  highly  uncertain  business  conditions, 

■  Value  the  feasibility  and  financial  viability  of  each  strategic  decision 
pathway, 
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Prioritize  pathways  or  projects  based  on  qualitative  and  quantitative 
metrics, 

Optimize  strategic  investment  decisions  by  elevating  different  decision 
paths  under  certain  conditions  or  determine  how  a  different  sequence 
of  pathways  can  lead  to  the  optimal  strategy, 

Time  effective  execution  of  investments  and  find  the  optimal  trigger 
values  and  cost  or  revenue  drivers,  and 

Manage  existing  or  develop  new  options  and  strategic  decision 
pathways  for  future  opportunities.  (Mun,  2005,  p  3) 
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5.0  Case  Study:  Open  Architecture  and 

Sustaining  Engineering  on  the  AEGIS  Platforms 


In  an  audit  conducted  by  a  management  consulting  firm,  sustaining 
engineering  in  the  AEGIS  system  was  identified  as  an  area  that  could  be 
reengineered  to  deliver  high  value.  In  a  further  analysis  by  NPS  researchers,  the 
AEGIS  software  maintenance  and  upgrade  process  was  subsequently  flagged  as  an 
area  where  OA  would  have  a  significant  impact.  This  section  provides  the 
background  information  necessary  for  the  proof-of-concept  case  study  by 
introducing  relevant  concepts:  distance  support  maintenance  solutions,  the 
component  business  model,  sustaining  engineering,  and  AEGIS  software 
maintenance  and  upgrade  processes. 

5.1  Distance  Support  Maintenance  Solutions 

According  to  the  2006  Distance  Support  Policy  released  by  the  Chief  of  Naval 
Operations,  distance  support  is  rapidly  becoming  “the  Fleet’s  principal  web-based 
readiness  enabler”  (Chief  of  Naval  Operations,  2006b,  p  10).  The  current  distance 
support  system,  at  a  minimum,  “combines  people,  processes,  and  technology  into  a 
collaborative  infrastructure  without  regard  to  geographic  location”  (2006b,  p  20).  It 
enables  ships  to  be  underway  for  several  months  and  communicate  with  shore- 
based  sites  to  fix  software  and  hardware  problems  that  occur  onboard  and, 
hopefully,  resolve  them  without  pulling  into  a  foreign  port  or  returning  to  the 
shipyards. 

In  the  future,  distance  support  will  also  include  shore-based  monitoring  of 
systems,  in  much  the  same  way  that  cars  sold  in  2006  and  2007  can  communicate 
with  central  databases  and  give  a  report  of  their  technical  status.  This  form  of 
distance  support,  called  remote  monitoring  and  notification,  in  a  possible  form  of 
procedure  for  shipboard  operations,  is  displayed  in  Figure  1 1  below. 
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Figure  11.  Remote  Monitoring  and  Notification 


As  seen  in  the  figure  above,  shipboard  information  is  constantly  monitored  in 
the  “data/information  acquired  phase,”  which  then  relays  the  information  to  the 
shore-side  server  for  diagnostics  and  assessments  of  trends  and  material  readiness 
If  there  is  a  problem  with  the  system’s  maintenance,  and  risk  recommendations  are 
made,  documented,  and  then  analyzed  in  metrics,  the  ship  is  notified.  If  there  is  no 
detected  issue,  then  the  monitoring  cycle  will  repeat  and  send  a  clean  report  to  the 
ship. 


When  distance  support  is  used  correctly,  it  complements  the  tenets  of  OA 
quite  effectively — if  upgrades  and  the  necessary  changes  can  be  made  to  an  open 
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system.  It  allows  for  modularity  and  design  reuse  that  can  be  distantly  monitored  and 
repaired  rather  than  requiring  a  visit  to  port  for  the  problem  to  be  fixed.  Additionally, 
as  illustrated  in  the  figure  above,  remote  monitoring  can  provide  automation  in  the 
process  of  upgrades  and  repairs;  and  automation,  in  most  cases,  leads  to  a 
decrease  in  the  necessary  number  of  employees. 


5.2  Component  Business  Model 

In  an  analysis  conducted  for  Program  Executive  Office,  Integrated  Warfare 
Systems  (PEO  IWS),  IBM  used  its  CBM  model  to  identify  sustaining  engineering  as 
an  area  that  could  be  significantly  reengineered  to  deliver  greater  value.  CBM,  a 
tool  developed  by  IBM,  identifies  opportunities  across  an  enterprise  for  innovation 
and/or  improvement”  (Pavlick,  2005,  p.  7).  CBM  breaks  down  the  enterprise  into 
business  components  consisting  of  resources,  people,  and  technology  that  have  the 
necessary  information  to  deliver  value  from  functional  performance.  Building 
component  models  (business  maps)  allow  managers  to  frame  decisions  on  a 
broader,  organizational  level  and  help  identify  areas  offering  the  greatest  opportunity 
for  innovation/improvement.  Each  component  encompasses  five  dimensions: 

■  A  component’s  business  purpose  is  the  logical  reason  for  its  existence 
within  the  organization,  as  defined  by  the  value  it  provides  to  other 
components. 

■  Each  component  conducts  a  mutually  exclusive  set  of  activities  to 
achieve  its  business  purpose. 

■  Components  require  resources:  the  people,  knowledge  and  assets  that 
support  their  activities. 

■  Each  component  is  managed  as  an  independent  entity,  based  on  its 
own  governance  model. 

■  Each  business  component  provides  and  receives  business  services, 
similar  to  a  standalone  business.  (IBM,  2005) 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


-21  - 


CBM  provides  a  framework  for  organizing  components  by  competency  and 
accountability  level,  as  seen  in  Figure  12.  The  framework  enables  executives  to 
envision  how  business  activities  might  function. 

Figure  12.  CBM  Framework 

(Pohle,  G.,  Korsten,  P.,  Ramamurthy,  S.,  2005,  p  7.) 
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•  Individual  business  modules  that  play  a 
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Key: 


■  Direct:  Provides  strategic  direction  and  corporate  policy  to  other  components.  Also  facilitates 
collaboration  with  other  components. 

■  Control:  Serves  as  checks  and  balances  between  “direct”  and  “execute”  levels.  Monitors 
performance,  manage  exceptions  and  act  as  gatekeepers  of  assets  and  information. 

■  Execute:  Provides  business  actions  driving  value  creation  in  enterprise.  Processes  assets 
and  information  used  by  other  components  or  end  customer. 


The  component  map  provides  a  basis  for  developing  strategic  and  operating 
insights  for  the  business.  By  gauging  the  relative  business  value  of  different  areas 
of  the  map,  executives  determine  which  components  demand  immediate  attention. 

In  addition,  “hot”  components  are  revealed  representing  the  greatest  economic  value 
based  on  pre-defined  attributes.  There  are  three  phases  for  the  CBM  framework,  as 
shown  in  Figure  13. 
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Figure  13.  Phases  of  CBM  Analysis 

((Pohle,  G.,  Korsten,  P.,  Ramamurthy,  S.,  2005,  p  10) 
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In  the  architecture  phase,  the  goal  is  to  identify  gaps  between  the  To-be 
vision  of  the  componentized  business  and  the  As-is  representation  of  how  the 
organization  presently  organizes  people,  processes  and  technology.1  The 
organization  subsequently  decides  how  to  close  the  gaps  in  the  last  investment 
phase. 


5.3  Sustaining  Engineering 

With  the  CBM  tool,  sustaining  engineering  in  software  maintenance  and 
upgrade  was  flagged  as  an  area  for  innovation  and/or  improvement.  Figure  14  is  the 
final  component  business  map  for  the  AEGIS  weapons  system;  its  “hot”  components 
are  represented  by  a  star.  IWS  PEO  selected  three  criteria  to  identify  those  “hot” 


1  To  capture  the  full  scope  of  the  firm’s  current  capabilities  and  market  positioning,  this  “as-is”  representation  must  be  firmly 
grounded  in  empirical  data,  such  as  organization  charts,  cost  drivers,  application  portfolios,  technology  investments,  key 
performance  metrics  and  existing  processes. 


VRAtVIANTIA  PER  SClfNr;^ 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


-23- 


components:  investment  of  total  budget  (green),  number  of  efforts  required  for  the 
task  (yellow),  and  color  of  money  (orange)  (Shannon,  2006,  p.  11). 

SE  has  a  medium  percentage  of  the  PEO  IWS  budget,  a  high  number  of 
efforts  (greater  than  six),  and  two  colors  of  money  involved.  The  colors  of  money,  or 
the  money  which  is  authorized/appropriated  and  used  for  specific  acquisitions,  are  in 
the  areas  of  Operation  and  Maintenance  Navy  (OMN)  and  Ship  Building  and 
Conversion  Navy  (SON).  The  horizontal  axis  in  Figure  14  represents  a  key 
competency,  or  one  which  requires  similar  skills  and  capabilities,  while  the  vertical 
axis  represents  accountability  levels.  SE  is  the  “System  Sustainment  and  Disposal” 
competency,  in  which  the  “executing”  branch,  the  branch  that  does  the  work,  is 
accountable. 

Additional  secondary  research  revealed  that  as  much  as  80%  of  the  total 
lifecycle  costs  of  a  system  are  incurred  in  the  Operations  and  Support  (O&S)  phase 
of  a  system.  Weapons  system  sustainment  consumes  about  “80  percent  of  logistics 
resources,  or  approximately  $64B  per  year,”  according  to  an  article  published  in 
Program  Managers  Magazine  (Kratz,  Fowler,  &  Cothran,  2002,  p.  2).  Given  that  a 
large  factor  of  the  total  lifecycle  costs  is  in  this  O&S  phase,  it  is  as  crucial  that  SE 
become  much  more  efficient  as  the  CBM  report  implied. 
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Figure  14.  AEGIS  CBM  Map 

(Shannon,  2006,  p.  14) 
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5.4  AEGIS  Software  Maintenance  and  Upgrade  Processes 

The  AEGIS  software  maintenance  and  upgrade  process  is  very  complex.  It 
involves  a  large  number  of  processes  in  four  main  phases:  requirements  definition, 
design,  test  and  implementation/installation.  The  entire  AEGIS  software  upgrade 
lifecycle  is  intended  to  take  18  months,  but  typically  takes  closer  to  24  months  due  to 
problems  found  during  the  testing  phase  or  failure  of  certifications.  This  software 
maintenance  and  upgrade  process  involves  many  sub-processes  in  each  one  of  its 
main  processes.  These  sub-processes  may  or  may  not  impact  the  rest  of  the 
processes  in  the  software  maintenance  and  upgrade  process.  The  fact  that  some  of 
these  sub-processes  may  be  able  to  function  in  a  stand-alone  capacity  makes  the 
analysis  of  the  software  maintenance  and  upgrade  process  very  difficult. 

The  software  maintenance  and  update  process  takes  place  in  two  primary 
areas:  on-ship  and  off-ship.  The  on-ship  portion  takes  place  aboard  AEGIS- 
equipped  US  Naval  vessels  and  is  conducted  by  Surface  Warfare  fleet  personnel 
and  various  support  personnel,  including  contractors.  The  on-ship  portion  is 
responsible  for:  identifying  problems  not  found  in  the  testing  phase  of  the  process, 
installation  and  on-ship  testing  of  the  fielded  AEGIS  software  update.  Two  different 
departments  detect  AEGIS  equipment  and  software  failures  and  analyze  their  effects 
on  mission  capability. 

Alternatively,  the  off-ship  portion  of  the  process  takes  place  at  the  Naval 
Surface  Warfare  Center,  Dahlgren,  VA.  This  primarily  deals  with  the  requirements- 
definition  phase,  the  design  phase  and  the  testing  phase.  Aggregated  AEGIS 
software  maintenance  and  update  processes  are  shown  in  Figure  15. 
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Figure  15.  AEGIS  Software  Maintenance  and  Update  Process  (Aggregate 

Level) 

AEGIS  WEAPONS  SYSTEM 
SOFTWARE  UPDATE 
PROCESS 

2/24/2007 
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A  number  of  sub-processes  are  required  in  the  off-ship  software  maintenance 
and  update  process,  as  seen  in  Figure  16. 


Figure  16.  Off-ship  Sub-processes 
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6.0 


Case  Study  Results 


The  KVA+RO  Framework  was  used  to  calculate  the  potential  impact  of  OA  on 
AEGIS  software  maintenance  and  upgrade  processes.  In  a  multi-phased  approach, 
analysis  was  first  conducted  for  one  ship  and  then  scaled  up  to  include  the  entire 
AEGIS  fleet  of  84  ships.  Data  used  in  both  analyses  was  derived  from  interviews 
with  Subject-matter  Experts  (SME’s),  surveys  and  secondary  research.2 

KVA  methodology  was  applied  to  on-ship  and  off-ship  processes  in  these 

steps: 

1 .  Identify  core  processes  and  sub-processes. 

2.  Establish  common  units  and  level  of  complexity  to  measure  learning 
time. 

3.  Calculate  learning  time  (i.e.,  knowledge  surrogate)  to  execute  each 
sub-process. 

4.  Designate  sampling  time  period  long  enough  to  capture  representative 
sample  of  the  core  processes’  final  product  or  services  output. 

5.  Multiply  learning  time  for  each  sub-process  by  number  of  times  sub¬ 
process  executes  during  sample  period. 

6.  Calculate  cost  to  execute  knowledge  (learning  time  and  process 
instructions)  to  determine  process  costs. 

7.  Calculate  ROK  (ROK=  Revenue/Cost)  and  ROI  (ROK=  Revenue- 
Cost/Cost). 

Assumptions  used  in  the  case  analysis  included: 


2  Collecting  accurate  data  for  KVA  analysis  was  challenging  given  the  complex  software  maintenance  and 
upgrade  processes,  along  with  the  large  number  of  people  involved.  Only  a  few  SMEs  understand  the  full 
complexity  of  processes.  In  addition,  outputs  and  learning  time  associated  with  each  process  and  sub¬ 
process  are  not  documented.  This  is  coupled  with  the  confusion  that  occurs  between  learning  time  and  time 
spent  in  a  Navy  training  course  to  learn  the  job.  The  Navy  training  courses  are  often  of  a  uniform  length  of 
time,  no  matter  the  complexity  of  job,  and  subject-matter  experts  often  confuse  these  training  times  with 
actual  learning  time.  There  is  also  a  need  to  separate  the  time  spent  in  a  Navy  training  course  or  school 
learning  the  specific  process  and  time  spent  learning  other  skills. 
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■  Use  of  middleware  was  necessary  until  Category  4  OACE  level  could 
be  reached. 

■  No  process  would  become  fully  100%  automated. 

■  One  employee  would  always  be  on-hand  as  a  supervisor  to  even  a 
mostly  automated  process. 

■  “Average  Time  to  Complete”  for  the  “New  Software  Version  Fielded  to 
Units”  was  estimated  to  be  15  minutes  using  the  distance  support 
concept. 

■  “Replacement  Technology”  would  be  used  instead  of  “Additive 
Technology.” 

■  Development  costs  were  not  included  because  they  are  distributed 
throughout  the  lifecycle  of  system. 

6.1  CASE  STUDY  RESULTS:  KVA  ANALYSIS 

Results  from  the  KVA  analysis  suggest  that  OA  could  have  a  significant 
impact  on  software  maintenance  and  upgrade  processes,  as  seen  in  Figure  17 
below.  Software  updates  are  available  via  a  push  or  pull  method  with  OA.  In  the 
pull  method,  the  user  downloads  and  installs  updates.  Alternatively,  in  the  push 
method,  software  is  pushed  to  the  network  node  remotely,  thereby  reducing  onsite 
personnel  while  speeding  up  the  upgrade  process.  Software  updates  are  available 
through  a  secure  link  provided  by  Operational  Readiness  Test  System  Tech  Assist 
Remote  Support  (ORTSTARS)  in  the  latter  scenario. 

New  software  updates  could  be  fielded  to  the  ship  through  ORTSTARS  in 
either  method,  resulting  in  reduced  cycle-time  fielding  new  software  to  its  shipboard 
configuration.  Remote  diagnostics  could  also  perform  the  functions  involved  in  the 
“Combat  System  Integration  Test,”  further  reducing  cycle-time.  Software  fielding 
through  distance  support  and  the  push/pull  method  would  also  reduce  the  number  of 
personnel  required  to  field  the  software  to  the  unit — from  three  employees  to  one 
employee.  The  one  process  executor  would  still  remain  available  to  oversee  the 
process  and  resolve  any  issues,  via  distance  support,  that  the  ship  may  encounter 
once  the  software  has  been  fielded  in  its  shipboard  configuration. 
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Figure  17.  OA  Enabled  Software  Update 
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In  particular,  OA  consolidates  four  sub-process  (“Software  Anomaly 
Detected,”  “Cause  of  Anomaly  Determined,”  “Software  Bug  Report 
Submitted,”  and  “New  Software  Version  Fielded  to  Units”)  into  two  (“Remote 
Diagnostics  Detect/Fix  Anomaly”  and  “Remote  Diagnostics  Submit  Software 
Bug  Report  for  Anomaly”): 

■  Remote  Diagnostics  Detect/Fix  Anomaly.  Through  ORTSTARS,  a 
remote  diagnostic  identifies  a  software  anomaly  before  an  operator 
onboard  identifies  the  anomaly.  Circumstances  surrounding  the 
anomaly  are  recorded  and  compared  to  similar  Computer  Program 
Change  Requests  (CPCRs)  managed  in  the  ACCESS/STARSY 
database.  If  a  CPCR  is  found  closely  matching  the  anomaly  detected, 
the  remote  diagnostics  could  then  take  appropriate  actions  already 
listed  in  the  ACCESS/STARSY  database  to  fix  the  anomaly. 

■  Remote  Diagnostics  Submit  Software  Bug  Report  for  Anomaly. 
Through  ORTSTARS,  the  software  bug  report  could  be  submitted  real¬ 
time  through  a  secure  link.  With  no  human  interpretation  required,  the 
software  bug  report  would  give  a  more  accurate  representation  of 
circumstances  surrounding  the  anomaly.  Although  the  process  still 
retains  some  human  intervention  as  one  process  executor  would  still 
oversee  the  process,  OA  plus  remote  diagnostics  could  drastically 
reduce  cycle-time  by  submitting  the  software  bug  report. 

Based  on  KVA  analysis,  implementing  OA  could  result  in  substantial  cost 
savings,  optimal  ROI  and  increased  benefits.  As  seen  in  Table  3,  costs  would 
decrease  $365,105  per  ship.  If  OA  was  applied  to  all  ships,  costs  would  decrease  by 
$26,543,825  per  year.  ROI  increases  from  69%  to  789%  for  one  ship.  And  for  all 
ships,  ROI  increases  from  320%  to  72,287%.  Potential  revenues  (benefits)  for  one 
ship  increases  $2,488,179  to  $3,837,931  and  if  OA  is  applied  to  all  ships,  revenues 
increase  $209,007,032  to  $322,386,181 . 
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Table  3.  Costs  for  One  Ship 


Process/ Revised  Process 

As-is 

To-be 

Potential 

Savings 

Software  Anomaly  Detected 

New  Release  Fielded  (Push  to  Ship  via  Distance 
Support) 

$14,301 

$7,150 

Cause  of  Anomaly  Determined 

Remote  Diagnostics  Detect/Fix  Anomaly 

$251,597 

$50,319 

Software  Bug  Report  Submitted 

Remote  Diagnostics  Submit  Software  Bug  Report  for 
Anomaly 

$1,430 

$1,430 

Anomaly  Verified 

$17,021 

$17,021 

Anomaly  Appended  to  Working  List  of  Known  Issues 

$1,307 

$1,307 

Workaround  Developed 

$17,021 

$17,021 

New  Software  Version  Developed 

$236,750 

$236,750 

Known  Anomalies  are  Resolved 

$100,639 

$100,639 

New  Software  Version  Fielded  to  Units 

$156,840 

$163 

Totals 

$796,907 

$431,802 

$365,105 

Table  4.  Costs  for  All  Ships 


Process/Revised  Process 

As-is 

To-be 

Potential 

Savings 

Software  Anomaly  Detected 

New  Release  Fielded  (Push  to  Ship  via  Distance 
Support) 

$14,301 

$7,150 

Cause  of  Anomaly  Determined 

Remote  Diagnostics  Detect/Fix  Anomaly 

$251,597 

$50,319 

Software  Bug  Report  Submitted 

Remote  Diagnostics  Submit  Software  Bug  Report  for 
Anomaly 

$1,430 

$1,430 

Anomaly  Verified 

$17,021 

$17,021 

Anomaly  Appended  to  Working  List  of  Known  Issues 

$1,307 

$1,307 

Workaround  Developed 

$17,021 

$17,021 

New  Software  Version  Developed 

$236,750 

$236,750 

Known  Anomalies  are  Resolved 

$100,639 

$100,639 

New  Software  Version  Fielded  to  Units 

26,349,120 

$13,723 

Totals 

$26,989,187 

$445,362 

$26,543,825 
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Table  5.  ROI  (One  Ship) 


Process/Revised  Process 

As-is 

To-be 

Software  Anomaly  Detected 

New  Release  Fielded  (Push  to  Ship  via  Distance  Support) 

297% 

3874% 

Cause  of  Anomaly  Determined 

Remote  Diagnostics  Detect/Fix  Anomaly 

236% 

4605% 

Software  Bug  Report  Submitted 

Remote  Diagnostics  Submit  Software  Bug  Report  for  Anomaly 

2108% 

39636% 

Anomaly  Verified 

234% 

234% 

Anomaly  Appended  to  Working  List  of  Known  Issues 

1 1 88% 

1188% 

Workaround  Developed 

78% 

78% 

New  Software  Version  Developed 

-36% 

-36% 

Known  Anomalies  are  Resolved 

-41% 

-41% 

New  Software  Version  Fielded  to  Units 

-36% 

185408% 

Totals 

69% 

789% 

Table  6.  ROI  (All  Ships) 


Process/Revised  Process 

As-is 

To-be 

Software  Anomaly  Detected 

New  Release  Fielded  (Push  to  Ship  via  Distance  Support) 

33278% 

333681% 

Cause  of  Anomaly  Determined 

Remote  Diagnostics  Detect/Fix  Anomaly 

28133% 

395159% 

Software  Bug  Report  Submitted 

Remote  Diagnostics  Submit  Software  Bug  Report  for  Anomaly 

185334% 

3337713% 

Anomaly  Verified 

27943% 

27943% 

Anomaly  Appended  to  Working  List  of  Known  Issues 

108113% 

108113% 

Workaround  Developed 

14856% 

14856% 

New  Software  Version  Developed 

5277% 

5277% 

Known  Anomalies  are  Resolved 

4841% 

4841% 

New  Software  Version  Fielded  to  Units 

-68% 

185408% 

Totals 

320% 

72287% 

ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


-34- 


Table  7.  KVA  Revenue  Analysis  for  One  Ship 


Process/ Revised  Process 

As-is 

To-be 

Difference 

(As-is,  To-be) 

Software  Anomaly  Detected 

New  Release  Fielded  (Push  to  Ship  via  Distance  Support) 

$56,826 

$284,131 

Cause  of  Anomaly  Determined 

Remote  Diagnostics  Detect/Fix  Anomaly 

$845,629 

$2,367,761 

Software  Bug  Report  Submitted 

Remote  Diagnostics  Submit  Software  Bug  Report  for 
Anomaly 

$31,570 

$568,262 

Anomaly  Verified 

$56,826 

$56,826 

Anomaly  Appended  to  Working  List  of  Known  Issues 

$16,837 

$16,837 

Workaround  Developed 

$30,307 

$30,307 

New  Software  Version  Developed 

$151,537 

$151,537 

Known  Anomalies  are  Resolved 

$59,194 

$59,194 

New  Software  Version  Fielded  to  Units 

$101,024 

$303,073 

Totals 

$1,349,752 

$3,837,931 

$2,488,179 

Table  8.  KVA  Revenue  Analysis  for  All  Ships 


Process/ Revised  Process 

As-is 

To-be 

Difference 

(As-is,  To-be) 

Software  Anomaly  Detected 

New  Release  Fielded  (Push  to  Ship  via  Distance  Support) 

$4,773,407 

$23,867,035 

Cause  of  Anomaly  Determined 

Remote  Diagnostics  Detect/Fix  Anomaly 

$71,032,841 

$198,891,956 

Software  Bug  Report  Submitted 

Remote  Diagnostics  Submit  Software  Bug  Report  for 
Anomaly 

$2,651,893 

$47,734,069 

Anomaly  Verified 

$4,773,407 

$4,773,407 

Anomaly  Appended  to  Working  List  of  Known  Issues 

$1,414,343 

$1,414,343 

Workaround  Developed 

$2,545,817 

$2,545,817 

New  Software  Version  Developed 

$12,729,085 

$12,729,085 

Known  Anomalies  are  Resolved 

$4,972,299 

$4,972,299 

New  Software  Version  Fielded  to  Units 

$8,486,057 

$25,458,170 

Totals 

$113,379,149 

$322,386,181 

$209,007,032 
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6.2  CASE  STUDY  RESULTS:  Real-options  Analysis 

Three  potential  strategies  were  considered.  In  Strategy  A,  the  As-ls 
alternative,  there  are  really  no  strategic  options  available.  It  mandates  keeping  the 
system  As-is  and  letting  it  retire  over  time.  Therefore,  the  total  strategic  value  is  the 
net  present  value  at  $196M.  With  Strategy  B,  the  options  are  also  limited;  the  option 
to  wait  and  defer  is  not  valued,  and  the  total  strategic  value  is  also  its  net  present 
value,  valued  at  $995M.  Strategy  C  is  an  option  to  wait  and  defer  with  a  proof-of- 
concept  for  the  first  year.  After  this  initial  test  case,  there  is  the  potential  for  a  follow¬ 
up  option  to  expand  into  the  next  phase,  generating  a  total  net  strategic  value  of 
$1 ,236M.  This  significantly  higher  value  comes  in  the  form  of  being  able  to  wait  and 
defer  a  decision  until  risks  and  uncertainty  become  resolved  over  the  passage  of 
time,  events,  and  actions,  and  in  this  case,  the  proof-of-concept  results. 

There  is  an  option  to  abandon  the  methodology,  should  the  results  from  the 
proof-of-concept  prove  to  be  under-performing  expectations.  If  expectations  are  met, 
however,  there  is  still  the  option  to  expand  and  execute  the  next  To-be  phase. 
Finally,  in  comparison,  Strategy  D,  has  a  slightly  lower  uncertainty  and  volatility  (the 
average  volatility  is  slightly  lower  than  in  Strategy  C),  with  a  much  higher  total  net 
revenue  from  all  the  phases.  Its  total  strategic  value  is  valued  at  $1,482M,  higher 
than  that  in  Strategy  C. 

Table  9.  Real-Options  Valuation  Results:  Strategies  A-D 


Strategy  A 

Strategy  B 

Strategy  C 

Strategy  D 

AS-IS 

TO-BE 

TO-BE 

STRATEGIC  OPTION 

(1) 

(W/out  Changes  in 
Fielding  to  Units) 

(1,3, 4, 5, 6, 7) 

TO-BE 

(1,2, 3, 4, 5, 6, 7) 

Total  Strategic  Value 

$196M 

$995M 

$1.24B 

1.48B 

Volatility 

10% 

30% 

60% 

50% 

Total  Cost 

$208M 

$96M 

$30M 

$80M 

Key: 

1 .  As-is 


2.  Implement  remote  diagnostics/prognostics  through  ORTSTARS/Distance  support,  plus  the 
ability  of  the  crew  to  inject  a  trouble  report  through  ORTSTARS/Distance  support. 

3.  Number  2  plus  providing  software  updates  to  the  ship  on  media,  then  having  the  crew  install  with 
technician  DS  (remote)  help.  (Could  also  postulate  a  "sense  and  respond"  sort  of  thing,  in  which 
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a  "local"  tech  is  scheduled  to  the  ship  based  on  its  availability  and  the  update's  arrival.  Would 
count  on  local  assets,  not  travel  from  Dahlgren...) 

4.  Number  3  plus  notification  of  the  ship  that  the  update  is  available  for  download.  Ship  initiates 
download  and  installs  with  DS  help. 

5.  Number  4,  except  that  the  ship  is  notified  that  updates  are  available.  The  on-ship  operators  tell 
DS  they're  ready,  and  the  remote  tech  takes  control  and  installs  the  update. 

6.  Number  4  except  that  the  update  is  pushed  to  the  ship,  then  cached  until  operators  are  ready  to 
install.  Ship  installs  with  DS  assistance  if  needed. 

7.  Final  state  in  which  the  update  is  pushed  to  the  ship  and  installs  during  slack  time,  notifying  the 
ship  and  allowing  operators  to  say  "not  now." 


7,0 


Summary 


This  proof  of  concept  case  identifies  the  potential  benefits  of  open 
architecture  in  the  AEGIS  software  maintenance  and  upgrade  process.  Open 
architecture,  built  upon  the  tenets  of  open  design  principles  and  architectures,  can 
assist  the  Navy  in  becoming  a  more  agile,  modular  and  cost-effective  enterprise.  As 
shown  in  this  case  analysis,  implementing  OA  could  result  in  substantial  cost 
savings  and  optimal  return  on  investment.  In  addition,  the  KVA+RO  framework 
provides  decision-makers  with  a  systematic  approach  for  structured  analysis  to 
evaluate  those  benefits  and  potential  risks  of  differing  strategies  when  implementing 
open  architecture. 
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Appendix  A.  Distance  Support  Best  Practice 
Example 


Maintenance-free  Operating  Period  (MFOP)  and  Acoustic  Rapid  Commercial  Off-the-shelf 

(COTS)  Insertion  (ARCI) 

In  2005,  Naval  Sea  Systems  Command  (NAVSEA)  completed  a  pilot  program 
to  test  the  feasibility  of  a  Maintenance-free  Operating  Period  (MFOP)  on  the 
Acoustic  Rapid  COTS  Insertion  (ACRI)  System.  The  ARCI  system  was  designed  to 
replace  the  AN/BSY-1  and  the  AN/BQQ-5  on  the  fleet’s  in-service  submarines 
(688/6881/Trident/Seawolf)  (Lockheed  Martin,  2005).  ARCI  was  a  success  in  its  own 
right,  in  that  it  effectively  demonstrated  the  use  of  OA  with  COTS  technology  on  a 
large  scale  in  the  fleet  and  allowed  for  technology  insertion  and  refreshment  (2005). 

The  ARCI  MFOP  program  was  conducted  over  a  one-year  time  span,  and  it 
tested  the  use  COTS  technology  and  the  COTS  support  provided  to  design  ARCI  in 
such  a  way  to  enable  MFOP.  Four  platforms  participated  in  the  testing,  and  over  the 
course  of  one  year  no  maintenance  was  required  in  any  of  the  four.  One  resulting 
benefit  of  this  test,  for  the  purposes  of  this  research,  was  that  the  platforms 
implemented  distance-support  capabilities  into  the  ARCI  system  before  they 
conducted  the  test  (NAVSEA  Surface  Warfare,  2005).  Most  particularly,  the  following 
results  are  applicable  for  the  formulation  of  To-be  models  for  AEGIS  software 
maintenance  and  upgrade: 


A  database  of  maintenance  related  data  was  built  into  the  ARCI 
system  which  provides  the  capability  to  perform  statistical  analysis  of 
system  performance  and  improve  availability.  An  availability  correlation 
function  was  developed  to  monitor  system  parameters  and  make 
recovery  recommendations  to  system  operators  [...]  An  additional 
benefit  of  the  MFOP  Pilot  Program  was  to  develop  and  implement 
functionality  in  the  ARCI  system  which  further  enables  the  system  to 
be  supported  via  Distance  Support  initiatives.  (NAVSEA  Surface 
Warfare,  2005) 
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Using  the  advances  outlined  above,  the  To-be  models  were  formulated.  The 
basis  for  those  changes  was  grounded  in  research  that  has  proven  its  MFOP 
reliability  over  the  course  of  an  entire  year. 
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Appendix  B.  Off-ship  Sub-processes 


Inputs  and 

Requirements 

Fleet  inputs,  external  interface  requirements  and  new  system  requirements 
gathered. 

System  Design  Review 

Technical  review  conducted  to  evaluate  optimization,  correlation, 
completeness,  and  risks  associated  with  the  allocated  technical  requirements. 

Summary  review  of  system  engineering  process  producing  allocated  technical 
requirements  of  engineering  planning  for  next  phase  also  conducted  (DoD, 
1985). 

ECPs,  SCPs,  ICRs 

Required  software  changes  are  documented  in  Engineering  Change  Proposal 
(ECP),  Software  Change  Proposal  (SCP)  and  an  Interface  Change  Request 
(ICR). 

Approval  Process 

Change  proposals  and  requests  sent  through  an  approval  process  in  which 
the  SCP  awaits  Software  Configuration  Change  Board  (SCCB)  approval,  and 
the  ICR  undergoes  Integration  Working  Group  (IWG)  approval. 

Aggregate  approvals  sent  to  NAVSEA  program  management  for  approval. 

Design  Review 

First  step  in  design  phase  of  the  AEGIS  software  maintenance  and  update 
process. 

Process  includes  a  preliminary  design  review  (PDR)  and  a  Critical  Design 
Review  (CDR). 

Design  Walkthrough 

Process  includes  writing  and  inspecting  code,  unit  test  and  analysis  and  code 
debugging. 

PDS/SDD 

Preliminary  Design  Specification  (PDS)  and  Software  Design  Document 
(SDD)  produced. 

Develop  Test  Plan 

First  process  in  test  phase  of  software  maintenance  and  update  process. 

Test  plan  developed  using  test  specifications  and  test-case  design  process. 

Test  Procedures 

Test  procedures,  outputs  of  develop  test  plan  process,  later  utilized  in  test 
execution  and  data-analysis  process. 

Test  Readiness  Review 
Process 

Test  plan  and  test  procedures  reviewed  to  ensure  most  effective  test  process. 

Collaborative  testing  and  data  analysis  included  to  achieve  maximum 
efficiency. 

Identify/Resolve  Issues 

Assessment  of  any  CPCRs  conducted  for  possible  program  update  and  also 
the  certification  impact  of  any  CPCRs. 

Certification  Impact 
Decided 

If  any  of  the  CPCRs  are  determined  to  have  the  potential  for  a  high 
certification  impact,  then  the  program  must  be  updated  before  it  can  be  sent 
to  the  test  execution  and  data  analysis  portion  of  the  software  maintenance 
and  update  process.  If  the  CPCRs  are  not  determined  to  have  a  high 
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certification  impact,  the  software,  then,  is  sent  directly  to  test  execution  and 
data  analysis. 

Update  Program  (High 
Certification  Impact) 

Software  containing  CPCRs  determining  to  have  a  high  certification  impact 
must  be  updated.  This  includes  another  unit  test  and  analysis  and  an 
assessment  of  the  certification  retest. 

Updated  Program 

Software  sent  to  test  execution  and  data  analysis  after:  program  updated,  unit 
test  and  analysis,  and  assessment  of  certification  retest. 

Test  Execution  and 

Data  Analysis 

Software  tested  under  lab  conditions  to  detect  potential  problems  prior  to  fully 
fielded  software.  Tests  consists  of  three  sub-processes: 

A.  Software  Anomaly  Discovered 

A  software  anomaly  is  found  under  lab  conditions. 

B.  Anomaly  Documented  in  CPCR  Database 

A  CPCR  is  generated  for  anomaly  and  then  entered  into 
ACCESS/STARSY  database. 

C.  CPCR  Assessed 

The  CPCR  is  prioritized,  and  its  certification  impact  is  assessed.  The 
CPCR’s  operational  impact  is  also  assessed,  and,  if  possible,  a 
workaround  is  established. 

Document  Results 

Test  execution  and  data  analysis  results  from  software  maintenance  and 
update  process  documented. 

Conduct  Functional 

Area  Assessment 

Higher-level  analysis  to  prepare  software  for  certification  panel  review. 

Conduct  Certification 
Panel 

A  certification  panel  assesses  the  software’s  results  from  the  test  execution 
and  data  analysis  process  and  certifies  the  software  for  fielding. 

PAT/FQT 

Preliminary  Acceptance  Test  (PAT)  and  Functional  Quality  Testing  (FQT) 
conducted  on  software. 

Data  Analysis 

Data  collected  from  PAT  and  FQT  assessed  and  analyzed  in  preparation  for 
the  Lab  Combat  Systems  Integration  Test. 

Lab  Combat  System 
Integration  Test 

Process  includes  any  final  testing  that  occurs  in  the  lab  environment, 
including  any  software  trouble  reports  (STR)  that  are  collected.  CPSA 
Analysis  and  a  CPSA  report  are  gathered. 

Shipboard  Delivery 

New  software  is  fielded  to  operational  units  and  installed  by  teams  of 
contractors  and  support  personnel. 

Crew  briefs  and  training  conducted. 

Shipboard  Combat 
System  Integration  Test 

Software  fully  tested  for  shipboard  configuration,  functionality  and 
interoperability  with  all  combat  systems  already  on  the  ship. 
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2003  -  2007  Sponsored  Acquisition  Research 
Topics 


Acquisition  Management 

■  Software  Requirements  for  OA 

■  Managing  Services  Supply  Chain 

■  Acquiring  Combat  Capability  via  Public-Private  Partnerships  (PPPs) 

■  Knowledge  Value  Added  (KVA)  +  Real-options  (RO)  Applied  to 
Shipyard  Planning  Processes 

■  Portfolio  Optimization  via  KVA  +  RO 

■  MOSA  Contracting  Implications 

■  Strategy  for  Defense  Acquisition  Research 

■  Spiral  Development 

■  BCA:  Contractor  vs.  Organic  Growth 

Contract  Management 

■  USAF  IT  Commodity  Council 

■  Contractors  in  21st  Century  Combat  Zone 

■  Joint  Contingency  Contracting 

■  Navy  Contract  Writing  Guide 

■  Commodity  Sourcing  Strategies 

■  Past  Performance  in  Source  Selection 

■  USMC  Contingency  Contracting 

■  Transforming  DoD  Contract  Closeout 

■  Model  for  Optimizing  Contingency  Contracting  Planning  and  Execution 

Financial  Management 

■  PPPs  and  Government  Financing 

■  Energy  Saving  Contracts/DoD  Mobile  Assets 

■  Capital  Budgeting  for  DoD 

■  Financing  DoD  Budget  via  PPPs 

■  ROI  of  Information  Warfare  Systems 
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■  Acquisitions  via  leasing:  MPS  case 

■  Special  Termination  Liability  in  MDAPs 

Logistics  Management 

■  R-TOC  Aegis  Microwave  Power  Tubes 

■  Privatization-NOSL/NAWCI 

■  Army  LOG  MOD 

-  PBL  (4) 

■  Contractors  Supporting  Military  Operations 

■  RFID  (4) 

■  Strategic  Sourcing 

■  ASDS  Product  Support  Analysis 

■  Analysis  of  LAV  Depot  Maintenance 

■  Diffusion/Variability  on  Vendor  Performance  Evaluation 

■  Optimizing  CIWS  Life  Cycle  Support  (LCS) 

Program  Management 

■  Building  Collaborative  Capacity 

■  Knowledge,  Responsibilities  and  Decision  Rights  in  MDAPs 

■  KVA  Applied  to  Aegis  and  SSDS 

■  Business  Process  Reengineering  (BPR)  for  LCS  Mission  Module 
Acquisition 

■  Terminating  Your  Own  Program 

■  Collaborative  IT  Tools  Leveraging  Competence 


A  complete  listing  and  electronic  copies  of  published  research  within  the  Acquisition 
Research  Program  are  available  on  our  website:  www.acquisitionresearch.org 
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